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FOREWORD 




The major long-term objective of Work Unit JOBTEST is to investigate and evaluate 
a variety of concepts and procedures for the measurement of job performances. Emphasis 
will be placed on identifying those techniques that have both validity and utility in 
practical testing environments and that have generality across groups of tasks. 

JOBTEST I was concerned with the development of a “hands-on-equipment” work 
sample criterion in the area of automotive maintenance. This report describes the results 
of this work. In addition, information was gathered concerning the present level of 
performance of general vehicle repairmen, MOS 63C30 and 63C40. The research was 
performed and most of the report preparation completed while HumRRO was part of 
The George Washington University. 

The research reported here is from the first of a series of studies. Subsequent 
experimental work will concern other measurement techniques in relation to the present 
work sample criterion test. t 

JOBTEST I was conducted during 1968 at HumRRO Division No. 2, Fort Knox, 
Kentucky, under Dr. Donald F. Haggard as Director. The Work Unit Leader is Mr. John 
D. Engel. 

Military support for the Work Unit is provided by the U.S. Army Armor Center and 
by the U.S. Army Armor Human Research Unit; the Military Chief of the Unit is LTC 
John A. Hutchins. SP5 Robert Rehder of the Armor HRU served as research assistant in 
the study. 

HumRRO research for the Department of the Army is conducted under Contract 
DAHC 19-70-C-0012. Work Unit JOBTEST is conducted under Army Project 
2Q062107A712, Training, Motivation, and Leadership Research. 



Meredith P. Crawford 
President 

Human Resources Research Organization 
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SUMMARY AND CONCLUSIONS 



MILITARY PROBLEM 

The Enlisted Personnel Management System is one of the largest Army users of 
proficiency tests. These tests are used to help implement many Army personnel programs, 
such as the proficiency pay and MOS qualification programs. 

The Board of Inquiry on the Army Logistics System (Brown Board) and other 
surveys of maintenance activities have raised questions about the validity of proficiency 
measures used in the awarding of MOSs and proficiency pay for the automotive 
mechanic. It is considered likely that the method of proficiency measurement most 
commonly employed— paper-and-pencil examinations— limits the validity of proficiency 
assessment that may be achieved in evaluating certain types of tasks. 

RESEARCH PROBLEM 

The major long-term research problem in Work Unit JOBTEST is to study and 
evaluate a variety of concepts and procedures for the measurement of job performance. 
Emphasis will be placed on identifying those techniques that have both validity and 
utility in practical testing situations, and that have generality across groups of tasks. 

The first phase of the research, and the primary problem dealt with in this report, is 
the development of a relevant and reliable work sample criterion for the General Vehicle 
Mechanic. This criterion will be used as a standard in later research phases that will 
evaluate various measurement techniques. 

During the development of the criterion, a secondary objective was to collect 
information on the quality of performance of organizational maintenance by general 
vehicle repairmen. 



METHOD 

Work was begun by updating job information in a 1964 HumRRO analysis of job 
requirements for consolidated MOS 630, 631, 632 (Automotive Mechanic). This updated 
job requirements inventory was used as a basis for developing items for a “hands- 
on-equipment” work sample. 

A four-day proficiency test consisting of 33 sample exercises was constructed. The 
test included a diagnostic scoring procedure for use in scoring men on quality of 
performance. The exercises were individually performed on track and wheel vehicles in 
common use and were individually scored by experienced mechanics who had been 
trained in proper test administration procedures. 

The test was administered to a total of 38 organizational mechanics, drawn from all 
the organizational maintenance units at Fort Knox, Kentucky. In addition, a question- 
naire was used to obtain information on personnel data, organizational maintenance 
experience, experience on various vehicle systems, current job assignment, type and 
amount of training, and amount of supervision received on the job. 



RESULTS 

(1) The item analysis values for the test were found to be well within accepted 
ranges for this type of data analysis. These analyses dealt with the difficulty level of the 
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items and their ability to discriminate between subjects who were high and low on the 
total pool of items. 

(2) The total test appears to have a high degree of reliability (r=.82), indicating it 
should permit a high degree of accuracy of measurement when used as a criterion in 
evaluating other measurement techniques. 

(3) On the average, 60% of the exercises were successfully completed by the 38 
mechanics. 

(4) There was a moderate relationship between performance and length of 
experience. 

(5) There was no practical difference in performance among mechanics with dif- 
ferent types of training (e.g., service school, other school, on-the-job). 

(6) Lack of use and unfamiliarity with technical publications, and lack of skill in 
the use of special tools and equipment were significant correlates of low proficiency. 



CONCLUSIONS 

(1) The work sample criterion appears to be a reliable and job relevant measure that 
may be used as a standard for the evaluation of other measurement techniques. 

(2) The results of this study indicate that the development of a proficient, well- 
trained mechanic is extremely complex, and that a more detailed examination should be 
made in the areas of training for troubleshooting tasks (as these appear to be the most 
difficult tasks to perform) and training in the use of technical publications and test 
equipment. 
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INTRODUCTION 



MILITARY PROBLEM 

A primary reason for the use of proficiency tests in the U.S. Army is to support the 
implementation of certain portions of the Enlisted Personnel Management System. The 
Management System, in turn, was devised to implement certain portions of the mission of 
the Deputy Chief of Staff for Personnel, Department of the Army (DCSPER), specifically 
DCSPER’s responsibility for “development and administration of a military personnel 
management system, to include policies and programs for procurement, individual training 
(less foreign military training), education, retention, career development, distribution, 
promotion, and separation of military personnel . . .”(1) The present study is based on a 
requirement from DCSPER. 

One of the largest users of proficiency tests is the Enlisted Personnel Management 
System. A survey by the Board of Inquiry on the Army Logistics System (Brown Board) 
and other surveys of maintenance activities have raised questions about the validity of the 
proficiency measures and performance standards employed in the Enlisted Evaluation 
System’s MOS evaluation. 

These MOS evaluation results are used in numerous Army personnel programs, some 
examples of which are: 

(1) Proficiency pay programs: MOS evaluation is designed to provide incentives 
to keep qualified soldiers with critical occupational skills. 

(2) MOS qualification: MOS evaluation provides for the verification of each 
soldier’s job proficiency. 

(3) Secondary MOS qualification: MOS evaluation ensures that the soldier has 
maintained his job proficiency depth in effective assignment and utilization actions. 

(4) Reserve and National Guard: Annual MOS evaluations of Reserve 
Component Unit Personnel help ascertain the readiness posture of our civilian soldiers. 

(5) Promotion qualification: MOS evaluation is geared to determine the 
enlisted man’s promotion qualification. The use of the promotion qualification score as a 
criterion for promotion is mandatory. 

(6) Enlisted grade and MOS determination: MOS evaluation is used to 
determine enlisted grade and MOS for those officers and warrant officers who are 
released from active duty in their commissioned or warrant officer status and enlist in the 
Regular Army. 

In general, the Enlisted Personnel Management’s Enlisted Evaluation System needs 
job proficiency tests that have a high degree of validity and reliability, and that are as 
inexpensive to develop and administer as possible (consonant with standards of validity 
and reliability), in order to effectively implement several portions of the Army’s 
personnel program. 



AN OVERVIEW OF PROFICIENCY MEASUREMENT TECHNIQUES 

There are various types of proficiency measurement techniques that might be used 
to evaluate a man’s achievement. According to Glaser and Klaus (2), proficiency measure- 
ment techniques may be grossly categorized on the basis of their remoteness from actual 
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job performance. This remoteness may be due to differences in (a) the behavior elicited 
for measurement, (b) the eliciting stimuli themselves, or (c) both stimulus and behavior. 
In most instances, however, as the test stimuli become more remote from those found in 
the actual job situation, the responses elicited are likewise less similar to those found in 
job performance. 

Thus at one extreme along this continuum of remoteness is the measurement of 
proficiency during actual job performance. At the other extreme are measures (e.g., 
paper-and-pencil tests) that are not obviously similar to the criterion task, but assess 
performance at tasks that correlate with on-the-job behavior. 

Between these two extremes are test situations that (a) call for the performance of 
the actual job task outside the real job environment, or (b) attempt to simulate the job 
task while at the same time offering effective control of the factors that in “real” 
situations are likely to interfere with reliable and valid measurement. The four major 
segments along this continuum can be identified as (a) on-the-job measures, (b) work 
sample measures, (c) simulated-job measures, and (d) correlated-job measures. 

In principle, proficiency measurement should be accomplished during a man’s typical 
performance, under conditions generally present during day-to-day operations. This 
method, however, presents a number of problems. The degree of control that can be 
achieved in a job situation is generally less than satisfactory for obtaining reliable 
measurements. In addition, attempts to standardize the situation for proficiency- 
measurement purposes frequently introduce considerable artificiality into the situation. 
Finally, the consideration of committing large amounts of time, money, and men to the 
testing situation often makes this an impractical method of assessment. 

To reduce, to some extent, the problems involved in on-the-job measures, samples of 
the actual job tasks involved may be removed from the real job environment so they can 
be readily and reliably assessed. This type of proficiency measurement technique is 
referred to as a work sample test. Here, the individual performs the actual tasks but not 
in the real job environment. This technique is a close approximation to on-the-job 
measures, but it has some of the same drawbacks: It is costly, time consuming, and 
essentially impractical as a method of assessing large numbers of people. 

Because it is difficult to measure men’s proficiency during actual job and work 
sample situations, the job must be simulated in a controlled manner in order to produce 
a reliable and valid, yet practical method of performance assessment. The essence of task 
simulation is the design of test stimuli that will evoke joblike responses that can be 
measured objectively. This general category of simulated-job measures includes a variety 
of proficiency measurement techniques. Some of the most frequently employed measures 
use equipment mock-ups and simulators. 

An extreme position along the dimension of remoteness from job reality is repre- 
sented by tests measuring, not job behaviors themselves, but correlated-job behaviors— i.e., 
measures correlated with job behavior. These measures are the most remote from the 
actual job situation. The most widespread type of correlated-job measure is verbal 
response as used to assess skills that are substantially nonverbal. Examples of this type of 
proficiency measure are tests of job knowledge, vocabulary, and nomenclature used to 
evaluate performance at procedural and manipulative tasks. Other types of correlated-job 
measures are those that involve a deliberate modification in the response made so as to 
facilitate the recording and evaluation of responses. A common example of this kind of 
construction is multiple-choice paper-and-pencil tests that are used to measure the ability 
to produce appropriate responses by measuring ability to recognize them. Because they 
are easily constructed, inexpensive, and easily administered, paper-and-pencil tests of job 
knowledge are frequently used to evaluate an individual’s proficiency. 

However, tests measuring knowledge of technical information, tool nomenclature, 
technical vocabulary, or underlying theory may not relate to actual performance for some 
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tasks. Instead, they measure verbal knowledge about the job, and therefore assess 
behaviors which, at best, may be correlated only slightly with actual job behavior— 
especially if the job depends on motor and manipulative skills. 

It is thus likely that paper-and-pencil tests, the kind of proficiency measurement 
most commonly employed, limit the validity of the evaluation of proficiency for certain 
tasks. Two examples are summarized below. 

(1) The U.S. Army Enlisted Evaluation Center conducted a validation study of the 
paper-and-pencil evaluation test for Track Vehicle Mechanic, MOS 63C20 (Yellen, 3). 
Performance of duties in this MOS is heavily dependent on perceptual-motor skills. The 
criterion used in the study was the average co-worker rating rendered by three enlisted 
men for each of the 47 enlisted men in the validation sample. The following conclusions 
were drawn from the study: “(a) The total evaluation test had a validity coefficient of 
.21; (b) optimal weighting of the Broad Subject Matter Areas did not significantly 
increase the validity of the total evaluation test.” 

It should be noted that a validity coefficient r of .21 is not statistically 
significant, but even if it were, it is far too low for use in group measurement and grossly 
inadequate for use in individual measurement. 

One reason for the lack of validity could be that an examination can differ- 
entially affect scores because of factors unrelated to actual job performance. The test 
may lean heavily, for example, on the individual’s ability to understand test directions or 
on his speed in reading lengthy descriptive passages. The influence of verbal facility on 
the test score appears to distort proficiency estimates in such a way as to systematically 
penalize those with poor vocabularies and reading skills, and not to reflect their profi- 
ciency in tasks that do not depend directly on verbal skills. 

(2) The Enlisted Evaluation Center conducted a validation study of the paper- 
and-pencil evaluation test for Personnel Specialist, MOS 716.1 (Urry, Shirkey, and 
Nicewander, 4). In this MOS, performance of duties is heavily dependent on verbal and 
reasoning skills, in definite contrast to the previous case in which perceptual-motor skills 
were predominant. Again, the criterion used in the study was the average co-worker 
rating rendered by three enlisted men for each of the 55 enlisted men included in the 
validation sample. The following conclusions were drawn from the study: “(a) The total 
evaluation test had substantial validity— the validity coefficient was .50; (b) optimal 
weighting of BSMAs did not increase the validity of the total evaluation test.” 

It should be noted that a validity coefficient r of .50 is sufficiently high for 
practical application in group measurement. 

One reason for the validity of the test for Personnel Specialists may be the fact 
that the test leans heavily upon the same skills that are required in the job— both the test 
and the job rely directly on verbal and reasoning skills. Therefore, in this instance a 
paper-and-pencil test seems to be appropriate for measuring performance in the MOS. 




RESEARCH PROBLEM 

The long-range research problem of Work Unit JOBTEST is to study and evaluate a 
variety of concepts and procedures for the measurement of job performance. Emphasis 
will be placed on identifying thchje techniques which have both validity and utility in 
practical testing environments, and which have generality across groups of tasks. 

The primary research problem dealt with in this report was to develop a relevant 
and reliable work sample criterion, which may be used as a standard against which all 
techniques will be evaluated. A secondary aspect deals with obtaining information about 
the quality of performance of organizational maintenance by general vehicle repairmen. 
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APPROACH 



A two-step approach was taken in order to achieve the long-range research objective. 
This report deals with the first step or the immediate research objective, that is, the 
development of a reliable work sample for assessing on-job performance, since perform- 
ance on the job must serve as the criterion for the validation of other assessment 
techniques. 

The second step will be concerned with the research and development of a range 
of proficiency measurement techniques varying in degree of remoteness from the work 
sample. These techniques will be compared with the work sample standard (as 
described in this report) on the basis of reliability, validity, ease of manufacture, 
application, and cost. 



METHOD 



SUBJECTS 

The G3 Personnel Section at Fort Knox, Kentucky, assigned 38 organizational 
mechanics from the available manpower in the various organizational maintenance units 
there. The selection was further based on MOS Code, and every effort was made by 
G3 to obtain men with varying degrees of experience and training within the 63C30 
and 63C40 MOS Codes. Fifteen men held the MOS Code 63C30 and 23 the Code 
63C40 (see Table 1). The subjects were divided into four groups— two of G3C30s and 
two of 63C40s, each group being tested for four days. 



Table 1 

Number of Subjects Tested in Each MOS by Type of Training and 

Years of Experience 



Type of Training 




Experience (years) 




Total 


and MOS 


0-1 


1-5 


5-10 


10-15 


15 or More 


Service School 














MOS 63C30 


0 


0 


2 


0 


0 


2 


MOS 63C40 


0 


3 


2 


2 


0 


7 


Other School 


MOS 63C30 


1 


3 


1 


0 


0 


5 


MOS 63C40 


0 


2 


2 


3 


3 


10 


Service and 
Other School 


MOS 63C30 


1 


5 


0 


1 


0 


7 


MOS 63C40 


1 


0 


0 


1 


0 


2 


On-the-Job 


MOS 63C30 


0 


1 


0 


0 


0 


1 


MOS 63C40 


0 


2 


1 


0 


1 


4 


Total 


3 


16 


8 


7 


4 


38 



EQUIPMENT, MATERIALS, AND RESEARCH PERSONNEL 



Test problems were set into the following vehicles: 



Quantity 



Type 

M60 tanks 

M151A1 Vi-ton trucks 

M54A2 5-ton trucks 

M35A1 214-ton truck 

M88 recovery vehicle 

Ml 10 self-propelled artillery vehicle 

M113 armored personnel carrier 

M108 self-propelled artillery vehicles 



3 

6 

2 

1 

1 

1 

1 

3 



Total 18 



Two large maintenance shops, each with two indoor and three outdoor bays, were 
used as testing facilities. 

Each man was provided with a mechanic’s tool kit, relevant technical manuals, 
troubleshooting guides, and test equipment. Test item sheets, scoring sheets, and a 
biographical questionnaire were developed (copies of these items are shown in 
Appendices A through D). 

Ten mechanics, E5 or higher, with the MOS Code 63C30 or 63C40,were trained 
to be test administrators. In addition, one noncommissioned officer in charge (NCOIC), 
one warrant officer, and one civilian researcher formed the research team and were 
responsible for the testing. 



TEST DEVELOPMENT 

A task inventory was necessary, to serve as a basis for future comparisons and as a 
reference for the development and evaluation of new performance measures. Such a task 
inventory had beeft completed at HumRRO Division No. 2 in March 1964, and it was 
considered sufficiently detailed and inclusive to warrant updating and use for the present 
study. 1 

Three senior automotive mechanics, with ten or more years of experience, served as 
technical advisors to review and update the 1964 inventory, with the constraints that 
(a) only task categories pertaining to individual (as opposed to group) performance were 
to be considered; and (b) only job level ”2” tasks (defined in the task inventory as 
essentially journeyman level tasks) were to be included. 

These constraints delimit the study and focus on tasks to be undertaken by 
journeymen— the level at which the mechanic is first responsible for performance of the 
full spectrum of organizational maintenance tasks. 

In the review and alteration of the Smith inventory, each of the three technical 
advisors was given a copy of the complete inventory and asked to pencil in any revisions 
deemed necessary in his independent appraisal. The reports were then combined for 
comparison, and the technical advisors’ modal responses were annotated for each area. 

On the basis of the information gathered (5), activities to be tested were selected 
from three task categories: (a) troubleshooting, (b) corrective action, and (c) preventive 
maintenance. In addition, examination of the task inventory showed that approximately 

1 The analysis for the task inventory was performed by Dr. John P. Smith. 
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60% of the vehicle systems on which the individual journeyman mechanic worked 
contained troubleshooting tasks; 35% of the vehicle systems on which the individual 
worked contained corrective action tasks; and 5% of the vehicle systems on which the 
individual worked contained preventive maintenance tasks. 

Test problems were drafted by one automotive Warrant Officer and two E7 
mechanics. Each man drafted 11 test items that were based on tasks he chose because of 
his expertise in specific automotive areas. These test items were drafted using the 
information provided in the job requirements inventory and the following criteria: 

(1) All items were to cover individual (as opposed to group) task categories of 
the journeyman level mechanic. 

(2) Problems covering the various task categories were to be represented in 
approximately the same proportion as the task categories occur in the job requirements 
inventory. 

(3) Approximately half of the items were to deal with wheel vehicle mainte- 
nance, the other half with track vehicle maintenance. 

(4) The task which the item was based on should be critical. That is, 
(a) without proper maintenance of part or system, vehicle could not maintain “combat 
readiness” (i.e., the vehicle must be able to make a trip of approximately 85 miles in not 
more than 15 hours); (b) if malfunction were not properly identified and diagnosed, a 
costly part or system of the vehicle might be unnecessarily replaced. Only tasks desig- 
nated as “High” on criticality would be considered. 

(5) The problem should be one that often occurs in the field, that is, a problem 
of high or medium frequency. Frequency was defined as: 

Low: No more than once or twice a year in a battalion-size unit. 

High: More than six times a year in a battalion-size unit. 

(6) High-density vehicles or vehicle systems would be utilized in forming 

problems. 

These criteria were carefully considered in preparing each item, but not all of them 
were maximally met in each problem. The various criteria had to be weighed by the item 
writers in order to produce a representative sample of items which could be used to test 
as many people as possible within a given period of time and within the limits imposed 
by availability of equipment and personnel. 

A total of 33 items— 17 troubleshooting, 12 corrective action, and four preventive 
maintenance— were completed and sent to the Automotive Department of the U.S. Army 
Armor School to be reviewed for currency and technical accuracy. 



TEST ITEM FORMAT 

In order to standardize test administration and scoring procedures, each item was 
composed of two parts. The first part (Appendix B) presented the content. It began with 
a statement of the symptom (for troubleshooting problems) or the action to be 
performed (for corrective action and preventive maintenance problems); this section was 
read by the tester to the subject. The essential procedures or approved steps for 
completing the task were then listed, with each step keyed as either essential (E) or 
optional (O), according to the judgment of the technical experts. Also included were 
spaces for writing in any parts mistakenly identified for replacement, and for recording 
the subject’s previous experience with similar tasks. 

The second part (Appendix C) was a performance checklist common to all test 
items. It was headed by a block of four passing and four failing categories. If a subject 
correctly reported the malfunction or correctly performed the required action, he was 
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scored in the pass section, and his degree of pass was indicated by one of four descriptive 
statements. The same type of scoring was used if he failed the problem. These descriptive 
statements are: 



Pass: 

Used good procedures; knew what he was doing 
Had a pretty good idea, but some guess work 
Knew very little; probably just a good guess 
Knew very little; but used publication accurately 

Fail: 

Didn’t know enough to get started 
Started but gave up quickly 

Some knowledge of what to do but much guess work also 
Had fairly good grasp of the problem but failed 

The use of this scoring provided additional information on the quality of perform- 
ance which could be related to the Army’s training system, in terms of identifying areas 
for training emphasis. 

All of the test items were scored on this two-part form. In this way, testing and 
scoring were made fairly uniform. 



TEST ADMINISTRATOR TRAINING 

Ten experienced mechanics (E5s and E6s) were trained to administer the tests. The 
training period for the testers was three and one-half days, during which time vehicles 
were placed in the working bays and problems were set into them. The problems were 
then tried out by the mechanics so they could familiarize themselves with test adminis- 
tration procedures and typical problems that might be encountered. 



TEST ADMINISTRATION 

Before a subject was tested, he was asked to fill out a questionnaire (shown in 
Appendix D) that required information in the following areas: (a) personnel data, 
(b) organizational maintenance experience on track and wheel vehicles, (c) experience on 
various vehicle systems, (d) current job assignment, (e) type and amount of training, and 
(f) amount of supervision received on the job. 

The subjects then received a thorough briefing on the nature of the tests and test 
procedures, and were encouraged to do their best on each problem. It was emphasized 
that they were not being personally evaluated for the record, but that their scores would 
be used solely for research purposes. They were further encouraged to view the tests as a 
novel learning experience. 

During the first day and a half of testing, each man was given the 17 trouble- 
shooting problems (20 minutes per station). During the next day and a half, they were 
given the 12 corrective action problems (20 minutes per station for eight stations, and 30 
minutes per station for four stations) and during the last day, the four preventive 
maintenance problems (15 minutes per station). 

Each day, the individual was given a schedule of his route from station to station 
and was told which station to report to first. One vehicle was located in each of five bays 
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(or work areas) of two large motor-pool buildings. 2 At each station the pretrained tester 
read the specific directions for the problem to the subject. (The directions were printed 
on each test sheet.) The subject was further instructed to proceed as he normally would 
on the job, using the test equipment, tools, and publications located at each test station. 
Testers were instructed to avoid conversation with the men being examined and to give 
them no help beyond repeating instructions. However, the tester did act as a helper by 
cranking the engine, holding a light, turning switches, and so forth, when requested. 

Performance was observed by the tester and recorded on the checklist. At the end 
of the time period (20 or 30 minutes), an air horn was blown and the men moved to 
another station, in this way completing the prescribed number of tests. After each 
exercise, the tester completed the summary score sheet (shown in Appendix C) which 
contained pass-fail categories of performance and some 20 statements describing the 
errors of performance. This same overall testing procedure was used for all four groups. 

The test stations were situated in such a way that a man could not easily observe 
the activity at another test station. In addition, men in the four different test groups 
were selected, whenever possible, from different organizational maintenance units at Fort 
Knox, so that communication between groups was minimized. 

It is believed that the actual job tasks were meaningfully duplicated except in two 
cases where it was necessary to remove the power plants from the vehicles and hook 
them up to the power source with “slave” cables or “ground-hop” kits. 



TEST SCORES AND PERSONNEL DATA 

Each exercise was scored on a pass or fail basis. A man’s total score was simply the 
number of exercises he completed correctly out of the 33 on which he was tested. 

The supplementary data used in the analysis include: (a) General Technical (GT) and 
Motor Maintenance (MM) aptitude scores; 3 (b) number of years of maintenance 
experience; (c) type and number of maintenance schools completed; (d) current job 
assignment; (e) experience on different types of vehicles; and (f) supervision given on 
the job. 



RESULTS 



TEST CHARACTERISTICS 
Item Analysis 

While a preliminary item analysis was performed, it should be realized that item 
analysis for this criterion measure is not the same as for classical psychometric applica- 
tions, because the content of a job sample criterion is established by analysis of the job. 
In other applications of item analysis, for classical psychometric uses, the main purpose is 
establishing a homogeneous pool of items. Homogeneity in the pool of items is, in 
principle, inapplicable for work sample tests such as this criterion measure whose content 
is job sampling or measuring components of a job which may or may not form a 

2 The other vehicles were parked in a third motor-pool building and used as a back-up for vehicles 
which may have become defective during testing. 

3 Army Aptitude Area scores based on combinations of scores on the Army Classification Battery 
(ACB). 





